Do consensus meetings undermine the validity of assessment centres?

In this study the effects of latent-informal processes operating in assessment centre consensus meetings is investigated with a combination of qualitative and quantitative methods. Non-participative observation is carried out in several consensus meetings, and auditory recordings made in three of these. In an analysis of the transcript of a consensus meeting in one organization, evidence is found for several latent-informal processes. These include active attempts by assessors to persuade other assessors, and the group facilitator, to appoint candidates; the use of assessors’ general impressions of candidates in this persuasion process; and the active use of power derived from an assessors’ relative seniority in the organization. Evidence consistent with the use of seniority-derived power is also found in a quantitative analysis of the selection decisions made in consensus meetings about 413 candidates. The results of the study are considered in relation to the practical utility of consensus meetings, and it is concluded that the use of such meetings is difficult to justify

An application of the theory of planned behavior to student retention

Although student integration theory, a sociologically-based model, has been the dominant explanation for student drop-out from colleges for over 40 years, it has received only mixed empirical support in residential colleges and less in non-residential colleges. Psychological theories of active choice and behavior change offer an alternative explanation for drop-out. In research at a non-residential UK university, structural equation modeling was used in two separate studies to compare a model of student dropout based on student integration theory with a psychological model based on the theory of planned behavior (TPB). In the first study (N=633), a model including TPB variables and two key student integration theory variables (academic integration, and social integration) showed good fit to the data, Although all three TPB variables predicted intention to quit, neither of the two student integration theory variables did so. The TPB variables explained over 60% of the variance in student's intention to voluntarily withdraw from college before completing their studies, and intention to withdraw was associated with actual dropout behavior. In the second study (N=180), using alternative measures of student integration theory factors, a model including both student integration theory and TPB variables had acceptable fit, and over 70% of the variance in intention to quit was explained. But only the TPB variables predicted intention to quit significantly. The benefits of adopting a process-based psychological explanation to student retention are discussed

When effects are confounded, they cannot be interpreted: A study of confounding in assessment centre ratings

Background. Dimensions (e.g., communication skills, health awareness) measured through multifaceted measures hold a key position across a vast array of disciplines relevant to health and organisations. However, the measurement properties of dimensions have long been affected by confounds. The outcome of confounding in the context of multifaceted measurement is that the extent to which reliable variance in ratings reflects dimensions is difficult or impossible to establish. Nowhere have dimensions been scrutinized more closely than in the literature on assessment centres (ACs). In recent years, dimensions have enjoyed a revival in the AC literature and now feature as important, meaningful, and impactful variables of interest in a range of different studies published in leading journals (e.g., Journal of Applied Psychology, Personnel Psychology). However, none of these studies have fully accounted for confounding. As a consequence, the extent to which the effects found in these studies are attributable to dimensions, or to a multitude of other variables relevant to ACs, is unclear. This issue has implications for any context in which the aim is to measure dimensions using multiple samples of behaviour. Study Aim. Our aim is to disentangle the different effects in AC ratings in order to establish what role dimensions have in AC ratings as distinct from other effects in AC ratings (i.e., different samples, rating items, assessors, and exercises). Doing so would work towards informing the discipline about the "real" basis for AC ratings and, more widely, about the role of dimensions in multifaceted measures. Method. Ratings were collected from an operational AC used in a public institution. The AC under study was developed according to international guidelines. Data were subjected to a Bayesian random effects analysis. Results. Results suggested that when confounding is (properly) taken into account, AC ratings have little to do with dimensions. The lion's share of variance in AC ratings is accounted for by (a) a general performance factor and (b) exercise-related effects. Discussion. The findings of the current study call into question the interpretation of much of the "dimension revival" literature on ACs and the role of dimensions in organization-related studies. They suggest that if dimensions are the intended focus for assessment in ACs, an AC does not allow sufficient high fidelity behavioural sampling for such variables to manifest. In the light of these findings, we call for an alternative direction for AC research and practice

CHIRPED PULSE AND CAVITY FT MICROWAVE SPECTROSCOPY OF THE HCCH-2,6-DIFLUOROPYRIDINE WEAKLY BOUND COMPLEX

The microwave spectrum of the HCCH-2,6-difluoropyrine complex has been observed using a chirped pulse and conventional cavity-type Fourier transform microwave spectroscopy. The acetylene moiety forms a hydrogen bond to the nitrogen of the 2,6-difluoropyridine, and this structure is contrasted with several systems involving HCCH or CO$_{2}$ bound to pyridine or 2,6-difluoropyridine. The results of DFT calculations support the experimental observations and are reported as well. The chirped pulse spectrometer is new in our laboratory and is built in tandem with our cavity-type spectrometer with a design that allows for switching between the two modes of operation without having to break vacuum. Pertinent details of the spectrometer will also be given

3-d printed slit nozzles for Fourier transform microwave spectroscopy

3-D printing is a new technology whose applications are only beginning to be explored. In this report, we describe the application of 3-D printing to the facile design and construction of supersonic nozzles. The efficacy of a variety of designs is assessed by examining rotational spectra OCS and Ar-OCS using a Fourier transform microwave spectrometer with tandem cavity and chirped-pulse capabilities. This work focuses primarily on the use of slit nozzles but other designs have been tested as well. New nozzles can be created for $0.50 or less each, and the ease and low cost should facilitate the optimization of nozzle performance (e.g., jet temperature or cluster size distribution) for the needs of any particular experiment

The perceived nature and incidence of dysfunctional assessment center features and processes

The aims of this research are to identify, for the first time, the dysfunctional features and processes perceived to take place in assessment centers (ACs) from multiple perspectives (assessment center designers, assessors, and candidates) and to indicate how often these phenomena are observed to occur. Two surveys were conducted in this study. In the first, a wide variety of dysfunctional processes and events were identified, and in the second, many of these processes are reported to occur with regularity. Based on these findings it is proposed that ACs should be construed, researched, and managed not only as large-scale psychometric systems but also as complex administrative, social, and political events susceptible to a broad range of dysfunctional phenomena

Everything that you have ever been told about assessment center ratings is confounded

Despite a substantial research literature on the influence of dimensions and exercises in assessment centers (ACs), the relative impact of these two sources of variance continues to raise uncertainties because of confounding. With confounded effects, it is not possible to establish the degree to which any one effect, including those related to exercises and dimensions, influences AC ratings. In the current study (N = 698) we used Bayesian generalizability theory to unconfound all of the possible effects contributing to variance in AC ratings. Our results show that ≤ 1.11% of the variance in AC ratings was directly attributable to behavioral dimensions, suggesting that dimension-related effects have no practical impact on the reliability of ACs. Even when taking aggregation level into consideration, effects related to general performance and exercises accounted for almost all of the reliable variance in AC ratings. The implications of these findings for recent dimension- and exercise-based perspectives on ACs are discussed

Behavioural Assessments in Construction Procurement: A Bandwagon of Institutional Waste?

The drive to create integrated and collaborative project teams has seen the behavioural assessment of suppliers become increasingly common in construction procurement exercises. Within the stated objectives of this are the desire to procure supply partners with the right ‘collaborative working capabilities’ and ‘cultural alignment’. The belief in the benefits of behavioural assessments in procurement has become so prevalent as to be referenced in the Infrastructure Client Group’s ‘Alliancing Code of Practice’ published by HM Treasury in 2015. However, the spread of this resource intensive practice has occurred without published evidence that it increases the effectiveness of procurement objectives. The purpose of this study is to examine the efficacy and value of behavioural assessment practices commonly used in UK infrastructure procurement exercises. The analysis of the study draws on theories of organizational psychology and sociology as well as the industry experience of the co-authors. In doing so, the study addresses ARCOM’s 2018 central theme, ‘Balancing fragmentation and integration’. Importantly, the study addresses practices attempting to secure integration but which evidence suggests generate actual and potential waste. It is concluded that the practices commonly used in behavioural assessment in construction procurement have little validity - the degree to which available evidence supports inferences and judgements made from scores on assessment measures. Also, the practice of using a small sample of assessed individuals to predict the behaviour of an organization as a whole over the life of a project has no known evidential foundation. The study’s findings shed light on institutional pressures in the development and introduction of management policies and construction procurement practices and call for greater collaboration between behavioural scientists and construction management disciplines. Such collaboration can be used to critically examine change proposals that may go on to generate ‘institutional waste’

Cognitive reflection predicts real-life decision outcomes, but not over and above personality and decision-making styles

The Cognitive Reflection Test (CRT), designed to assess the ability to inhibit intuition to process a problem analytically, predicts people's performance in many normative judgement and decision-making tasks (e.g., Bayesian reasoning, conjunction fallacy and ratio bias). However, how the CRT predicts normative decision-making performance is unclear, and little is known about the extent to which the CRT predicts real-life decision outcomes. We investigate the role of the CRT in predicting real-life decision outcomes and examine whether the CRT predicts real-life decision outcomes after controlling for two related individual differences: the Big Five personality traits and decision-making styles. Our results show that greater CRT scores predict positive real-life decision outcomes measured by the Decision Outcome Inventory. However, the effect size was small, and the relationship became non-significant after statistically controlling for personality and decision-making styles. We discuss the limited predictive role of cognitive reflection in real-life decision-making outcomes, along with the roles of personality and decision-making styles. Copyright © 2015 John Wiley & Sons, Ltd

A comparative study of practitioner perceptions of selection methods in the United Kingdom

In this study, 476 participants, divided into occupational psychology- (OP), Chartered Institute of Personnel and Development- (CIPD), human resource management- (HRM) qualified, and layperson subgroups, provided their perceptions of the validity, fairness, and frequency-of-use of employee selection methods. Results of a mixed-effects analysis of covariance revealed that respondent qualification background predicted the degree to which participant validity perceptions were aligned with research-based estimates of validity (F[3, 29.39] = 20.06, p < .001, η2 = .67). Corrected pairwise comparisons suggested that perceptions of participants with CIPD and HRM backgrounds were not significantly more aligned with research estimates of validity than were the perceptions of laypeople. OP participant validity perceptions were significantly more aligned with research estimates than all other subgroups, (p < .03). Evidence was also found for some between-group consistency regarding frequency-of-use perceptions, but less between-group consistency was found vis-à-vis perceptions of fairness. Implications for decision making in employee selection are discussed